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Abstract: We generalize the point information gain (PIG) and derived quantities, i.e., point 
information gain entropy (PIE) and point information gain entropy density (PIED), for the case of the 
Renyi entropy and simulate the behavior of PIG for typical distributions. We also use these methods 
for the analysis of multidimensional dafasefs. We demonsfrafe fhe main properties of PIE/PIED 
specfra for fhe real dafa wifh fhe examples of several images and discuss furfher possible utilizations 
in ofher fields of dafa processing. 
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1. Introduction 

Measurement of relative informafion between fwo probabilify disfribufions is one of fhe mosf 
imporfanf goals of informafion fheory. Among many ofher concepfs, fhere are fwo fhaf are 
widely used. By far, fhe mosf widespread concepf is called fhe relafive Shannon enfropy or fhe 
Kullback-Leibler divergence. In fhis work, we use an alfernafive approach based on a simple concepf 
of enfropy difference insfead. By generalization of bofh concepfs from Shannon's approach fo Renyi's 
approach, we obfain the whole class of information variables that enable aiming for different parts of 
probabilify disfribufions and inferpref if as an invesfigafion of different parts of mulfifracfal sysfems. 

Despife fhe mafhemafical precision of fhe concepf of fhe Shannon/Renyi divergence, we use 
anofher concepf, fhe (Renyi) enfropy difference, for infroducfion of a value which locally defermines 
an informafion confribufion of a given elemenf in a discrefe sef. Even fhough fhere is no subsfanfial 
resfricfion on fhe usage of a sfandard divergence for calculation of fhe informafion difference upon 
eliminafion of one elemenf from a sef, for practical reasons, we used the simple concept of enfropy 
difference befween sefs wifh and wifhouf a given elemenf. The resulfed value has been called a poinf 
informafion gain E^’^ [1,2]. The goal of fhis arficle is fo examine and demonsfrafe some properfies of 
this variable and derive another quantities, namely a point information gain entropy and a point 
information gain entropy density E^. We also introduce the relation of all fhese variables fo global 
and local information in multidimensional data analysis. 
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2. Mathematical Description and Properties of Point Information Gain 


2.1. Point Information Gain and Its Relation to Other Information Entropies 

An important problem in the information theory is to estimate the amount of information gained 
or lost by refining, and approximate the probability distribution P by the distribution Q. The most 
popular measure used in the theory is the Kullback-Leibler (KL) divergence, defined as 

Dkl{p\\Q) = [inQ] = Sp(Q) - S{P), (1) 

/ 

where Sp(Q) is so-called cross-entropy [3] and S(P) is the Shannon entropy of distribution P. If P is 
similar to Q, this measure can be approximated by entropy difference 

AS(P,Q) = S(Q)-S(P). (2) 


Indeed, this measure does not obey as many theoretic-measure axioms as the KL-divergence. 
For instance, for P Q, we can still obtain AS{P,Q) = 0. Nevertheless, if P w Q and 
P Q, this value can be still a suitable quantity revealing some important information aspects of 
a system. The situation, when the distributions are approximative histograms of some underlying 
distributions P for n and (n -|-1) measurements, respectively, is particularly interesting. In this case, 
the entropy difference 

AS(P„,P„+i) = S(P„+i)-S(P„) (3) 

can be interpreted as an information gained by the (n -|- l)-th measurement. Naturally, P„ ^ P. 
When dealing with real complex systems, it is sometimes advantageous to introduce new information 
variables and entropies that capture the complexity of the system better, e.g., Hellinger's distance, 
Jeffrey's distance or J-divergence. There are also some specific information measures that have 
special interpretations and are widely used in various applications [4,5]. Two of the most important 
quantities are the Tsallis-Havrda-Charvat (THC) entropy [6], which is the entropy of non-extensive 
systems, and the Renyi entropy, the entropy of multifractal systems [7,8]. The latter is tightly 
connected to the theory of multifractal systems and generalized dimensions [9]. It is defined as 

= ^InEP/. a>0, (4) 

j 


where a is the Renyi coefficient and pj is the probability of occurrence of a phenomenon j in 
the discrete distribution. Limit a —> 1 recovers the Shannon entropy. Similar to the Shannon 
entropy, the Renyi entropy also has an operational meaning. Actually, it can be interpreted as the 
average information cost, when the cost of an elementary piece of information is an exponential 
function of its length [10]. Thus, changing the parameter a changes the cost of the information 
and therefore accentuates some parts of the probability distributions while suppressing the others. 
Thus, by taking into account the whole class of Renyi entropies, we get a new generalized class of 
information quantities. 

The point information gain T^ ^ of the i-th point was developed as a practical tool for assessment 
of the information contribution of an element to a given discrete distribution [11]. Similar to the 
Shannon entropy difference, it is defined as a difference of two Renyi entropies—with and without 
the examined element of a discrete phenomenon. Let us consider a discrete distribution of k distinct 
possible outcomes (e.g., different colors of pixels). Let us have a discrete distribution 


p = {Pi}U = 
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where n denotes the total number of elements in the discrete distribution and m, the number of 
elemenfs of f-fh phenomenon, i G {1,2, ...,k — l,k}, respecfively. Lef us denofe = rij for j 7 ^ i 

and = n, — 1. Then, fhe disfribufion wifh fhe omiffed i-fh phenomenon can be wriffen as 




n — 1 n — 1 


( 6 ) 


Hence, we may wrife fhe poinf informafion gain T* ^ as 


■p{0 _ 

‘-a. 


W.(pl'')-W,(P) = j^ln(EW''T)-T^ 
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7=1 f, 


where k is fhe fofal number of fhe phenomena in fhe discrefe disfribufion. In fhe resf of fhe fexf, we 
use fhe nafural logarifhm fo simplify calculafions. However, all compufafions have been performed 
wifh fhe usage of binary logarifhm which, for fhe Renyi enfropy and ifs derivafives, yields values in 
bifs. In confrasf fo fhe commonly used Renyi divergence [12-18], we use ri'^ for ifs relative simplicity 
and practical interpretation. Unlike the KL divergence, the Renyi divergence cannot be interpreted 
as a difference of cross-enfropy and enfropy of fhe underling disfribufion and compufafion becomes 
infracfable. As discussed above, for similar disfribufions, if sfill preserves ifs informafion values. 

Affer fhe subsfifufion for fhe probabilifies, one gefs fhaf 


r (0 _ 

A /y — 


1 — a 


In 


M=1 (n-iy 

yjOC 


— Cft(n) + 


\ — DC 


In 


H")‘ 

4=1 


( 8 ) 


a f/') 

where Ca{n) = ln(^^)i“" depends only on n. For n ^ 00 and F^ ^ 0, fhe whole enfropy 
remains finite (contrary to unconditional entropy, which has to be renormalized for continuous case 
(for defails, see Reference [7])). Therefore, we examine only the second term. When the argument of 
fhe logarifhm is close fo 1 , i.e., 

E]”)") =»E"*. 0 ) 

/=1 ;=1 

which leads fo fhe condition fhaf 




( 10 ) 


for given a, one can fhen approximafe fhe logarifhm by fhe Taylor expansion of fhe firsf order. 
Affer denoting 
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where we used the big O asymptotic notation. Let us note that the last term in Equation (8) is nothing 
else than the THC entropy [6,19]. Naturally, for very similar distributions, these two quantities are 
practically the same. This is due to the fact that, for large n, fhe omission of fhe poinf has no large 
impacf on fhe whole disfribufion. Consequenfly, an acfual value of paramefer a, which leads fo 
rescaling of probabilities, is more imporfanf fhan a parficular form of enfropy. 

We shall continue by utilizing the Renyi entropy due to its relation to the generalized dimension 
of mulfifracfal sysfems [20,21]. Lef us concenfrafe again fo fhe ferm 'D^'^. We can rewrife if as 


= 

Ly n 
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where we use fhe small co asympfofic nofafion. Specifically, provided a = 2, we obfain 
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(14) 


which explains why fhe dependency n, on T^*^ is approximafely linear. In general, point information 
gain is a monotone function of n,, respectively p;, for all possible discrefe disfribufions. Thus, if may 
be used as a quanfify of informafion gain between fwo discrefe disfribufions, which in fhe occurrence 
of one parficular feafure, differ. 

Lef us discuss an inferprefafion of fhe poinf informafion gain. We can rewrife Equation (8) as 
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We are inferesfed in fhe sifuafion when T^ ^ =0. Affer sfraighfforward manipulafions, we can gef rid 


(16) 


of In and power, so 
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Thus, we end wifh 


m = 




(17) 


(18) 


This shows fhaf T^^ = 0 holds for evenfs wifh average frequency. T^^ < 0 corresponds fo 
rare evenfs, while T^^ > 0 corresponds fo frequenf evenfs. Thus, in addition fo fhe definition 
of fhe quantify of fhe contribufion of each evenf fo fhe examined disfribufion, we also obfain fhe 
discriminafion befween poinfs which confribufe fo fhe fofal informafion of fhe given disfribufion 
under fhe sfafisfical assumpfion represenfed by a parficular a. This opens fhe quesfion on exisfence 
of fhe "optimal" disfribufion for fhe given a. 
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Then, the possible variants of such optimality arise subsequently: the first one can be defined 
as a disfribufion for which exacfly half of fhe value n, produces T^ ^ > 0 and fhe ofher half yields 
Ttt ^ < 0. The second one requires values T^ ^ fo be spaced equally Exisfence of such a disfribufion 
could be understood as anofher generalizafion of fhe concepf of fhe enfropy power [22,23], and we 
refer fhis question fo our fufure research. 

Wifh respecf fo fhe previous discussion and pracfical ufilizafion of fhis nofion, we emphasize 
thaf for real sysfems wifh large n, values T^^ are relatively small numbers for currenf numerical 
precision of common computers. Their further computer averaging and numerical representation 
lead to significant errors such as underflow and overflow (e.g.. Figure Ic). Af lower values a, fhe 
values T^ ^ are broadly separafed for rare poinfs, while, af higher values a, fhe resolution is higher for 
more frequenf dafa poinfs. Therefore, specfrum ri*^ vs. a is more advisable fo compute rafher fhan a 
single T^ ^ value af a chosen a. 



Figure 1. r„ /-transformations of the discretized Levy (a), Cauchy (b), and Gauss (c) distribution 
at a = 0.99. The deviation from the monotone dependency in the Gauss distribution is due to the 
digital rounding. 

2.2. Point Information Gain for Typical Distributions 

In Figure 1, we demonstrate T^ ^-transformations of fhree fhoroughly sfudied disfribufions—fhe 
Levy, Cauchy, and Gauss disfribufion (specified in Secfion 4.1). Mainly, Figure Ic shows averaging of 
digifal levels, which resulfs in mulfiple appearance of unique poinfs. This phenomenon is reduced 
wifh fhe increasing number of fhe poinfs in fhe disfribufion. Neverfheless, if does nof disappear in 
any real case. Thus, fhe monofone dependencies of n,, respectively p/, on fhe T^’^ are valid only af fhe 
approximafion fo an infinite resolufion in levels of values. 

Figure 2 shows disfribufion changes of fhe values T^ ^ wifh fhe increasing a-paramefer. For each 
parameter a, fhe elemenfs T^ ^ are enveloped by monofone increasing curves. For insfance, as devised 
in Equafion (14), fhe near linearify of fhe dependency of fhe number of elemenfs on fhe values T^ ^ af 
a = 2 is seen in Figure 2d. The differences between fhe disfribufions are expressed by fhe disfribufions 
of fhe values T^ ^ along fhe horizonfal axes. 
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Figure 2. -transformations of fhe discretized Levy distribution at a = {0.5,0.99,1.5,2.0,2.5,4.0} 

(from (a) fo (f)). 

2.3. Point Information Gain Entropy and Point Information Gain Entropy Density 

In the previous sections, we showed that is different for any n, and the dependency of 
these two variables is a monotone increasing function for all a > 0. Here, we propose new 
variables—a point information gain entropy (Hf) and point information gain entropy density (2^) 
defined by formulas 

= (19) 

M 

and 

= ( 20 ) 

M 

They can be understood as a multiple of the average point information gain and—under linear 
averaging—an average gain of the phenomenon j, respectively. 

The information content is generally measured by the entropy. The famous Shannon source 
coding theorem [24] refers to a specific process of transmission of a discretized signal and introduction 
of the noise. The Renyi entropy is one of the class of one-parametric entropies and offers 
numerous additional features over the Sharmon entropy [7,12,25] such as the determination of a 
generalized dimension of a strange attractor [20,21]. The universality of the generalized dimension 
for characterization of any distribution, whose regularity may be only coincidental, is still under 
dispute. However, the values Ha and characterize a given distribution for any a. Differences 
between distributions are expressed in counts along the axes r^. Therefore, independently of the 
mechanisms of the generation of the distributions, the values Ha/Ea can serve for the comparison of 
these distributions. It holds for any both parametric and non-parametric distributions. 
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The next question is whether the has some expected properties. In this aspect, we mention 
the facts observed upon examination of Equafion (12), which enable us fo rewrife if as 
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( 21 ) 


where fhe producf in fhe argumenf of fhe logarifhm in fhe second ferm is a producf of functions 
upper limifed by 1 and fhus again a function upper limifed by 1. From fhe previous analysis done for 
fhe we may conclude fhaf fhe poinf informafion gain enfropy densify (E^) inherifs properfies of 
Renyi enfropy, i.e., zooming properfies, efc. 

Similar fo Equafion (21), fhe poinf informafion gain enfropy (Ha) can be rewritten as 
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Again, fhe argumenf of fhe logarifhm in fhe second ferm is upper limifed by 1. The Ha also has 
properfies inherifed from Renyi enfropy, alfhough fheir mufual relation is more complicafed. 

3. Estimation of Point Information Gain in Multidimensional Datasets 


3.1. Point Information Gain in the Context of Whole Image 

Point information gain Ta,, introduced in Equation (7) was originally applied to the image 
enhancement [1,2]. A typical digital image is a matrix of x x y x n values, where x and y are 
dimensions of fhe image and n corresponds fo fhe number of color charmels (e.g., n is 1 and 3 for 
a monochrome and RGB image, respectively). In mosf cases, fhe infensify values are in fhe range 
from 0 fo 255 (an 8-bif image) or from 0 fo 4095 (a 12-bif image) for each color charmel. For any size 
and bif depfh of an image, we can compufe fhe global informafion (Algorifhm 1) provided by fhe 
occupied infensify bin i and evaluafe as a change of a probabilify infensify histogram after removing 
a poinf from fhis bin. 

For each paramefer a, fhe calculation of Ta^ helps fo find values of fhe intensifies wifh fhe 
identical occurrences and determine fheir disfribufion in (a sfrucfural parf of) fhe image. Thus, in 
general, fhe recalculations to Ta ^ can be considered as Look-Up Tables—intensities with the highest 
probabilities of occurrences in an image correspond fo fhe highesf (posifive) values Ta^ and fhe 
brighfesf intensifies in a Ta^-fransformed image and vice versa. Sometimes, mainly in fhe case of 
local informafion, due fo fhe fransformafion of fhe original values Ta ^ info an 8-bif resolufion, some 
levels Ta ^ are merged info one infensify level of fhe fransformed image. 
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Algorithm 1: Point information gain vector (Fa), point information gain entropy (H^), and 

point information gain entropy density (E^) calculations for global (Whole image) information 

and f 5 rpical hisfograms. 

Input: n-bin histogram h; a, where a > 0 A a 7 ^ 1 

Output: r«; H«; E^ 

1 p = h/sum(h); % explain the frequency histogram has a probability histogram p 

2 Fa = zeros(h); % create a zero matrix F^ of the size of the histogram h 

3 for f = 1 to n do 

4 h 2 = h; % create a vector h 2 identical to the histogram h 

5 if h 2 (f) 7 ^ 0 then 

6 h2(f) = h2(f) - 1; 

% if the bin i of the histogram h 2 is occupied, remove an element at the position i 

7 end 

8 p 2 = h 2 /sum(h 2 ); % calculate a probability histogram p 2 without the examined element 

9 fI'^ = log2(sum(p2.^a)/sum(p.^a)); 

% calculate F^^ as a difference of two Renyi entropies - with and without the examined element, 
respectively Equation {?)) 

10 end 

11 Hft = sum(h.*Fa); 

% calculate Ha as a sum of the element-by-element multiplication ofh and Ta{Equation (19)) 

12 Ea = sum(Fii); % calculate E^ as a sum of all unique values in la (Equation (20)); 

Everything is best visualized in Figures 3-4, which show the F^'^-transformations of fhe 
fexmos2.s512 image. The infenfion was probably fo creafe an image wifh a uniform disfribufion 
of infensifies. Provided fhe uniform infensify disfribufion, fhe oufpuf of fhe global -calculation 
would be only one value E^a , i.e.. Figure 3b would be unicolor. However, eighf original infensifies 
(Figure 3a) resulfed in five values Fg gg (i.e., local parfs) (Figure 4b,d). The defailed image analysis 
showed fhaf fhe number of occurrences is only identical for infensifies 32-224 and 96-128-192, i.e., 
fhere are five unique values of frequencies of infensify occurrences (Figure 4a). For a change, in 
fhe 4.1.07 image, fhe global Tg^g-recalculation emphasizes fhe unevenness of fhe background and 
shadows around a group of fhe jelly beans (Figure 5b). In conformify wifh fhe sfafemenf in fhe 
nexf-fo-lasf paragraph in Secfion 2.1, fhis principle also enables highlighting of rare poinfs in images 
wifh rich specfrum of infensifies, mainly af low a-values. The calculations using higher values a do 
nof poinf highlighf rare poinfs so infensively and fhe resulfing image is more smoofh. 
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Figure 3. -transformations of the texmos2.s512 image [26]. Original image (a) and information 
images calculated from the whole image (b), a cross around each pixel (c), and squares of the side of 
5,15, and 29 px, respectively, with the centered examined pixel (d-f). 
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Figure 4. Histograms of FQ^g-transformations of the texmos2.s512 image [26]. Original image (a), 
original values F[,‘gg calculated from the whole image (b), original values Fg gg calculated from a cross 
whose shanks intersect in the examined pixel (c), Fg gg-transformed images calculated from the whole 
image (d), and Fg gg-transformed images calculated from a cross around each pixel (e). Colors in the 
original and globally (whole image) transformed histograms correspond to the intensity levels with 
the identical frequencies of occurrences in the original image. 
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Figure 5. rg^g-transformations of the 4.1.07 image [26]. Original image (a) and information images 
calculated from the whole image (b), a cross around each pixel (c), and circles of the diameter of 5,17, 
and 30 px, respectively, with the centered examined pixel (d-f). 

3.2. Local Point Information Gain 

Since multidimensional datasets, as e.g., images, consist of special structures given by the 
pixel lattice, it can be also beneficial to calculate not only global information gain, but also local 
information gain in some defined surroundings (Algorithm 2). The local information is defined 
after removing an element from the bin i where the element lies in the center of the surroundings, 
which creates the intensity histogram. The choice of the local surroundings around pixels is 
specific for each image. However, we do not have any systematic method for comparison of 
suitability of different surroundings around the pixels. The suitability of the chosen surroundings 
depends obviously on the process by which the observed pattern or other distribution was 
generated. According to our knowledge, the choice of the appropriate surroundings on the basis 
of known image generation was studied only for cellular automata [27-29]. This makes the study 
of the local information very interesting because it outlines another method for recognition of 
the processes of self-organization/pattem formation [30]. In this article, we confine ourselves 
to the usage of the local information for better understanding of both the limitation of the 
method of the T^ ^-calculation and the local information itself. The cross, square, and circular 
surroundings around each pixel are demonstrated on three different standard images—texmos2.s512 
(monochrome, computer-generated, unifractal), 4.1.07 (RGB, photograph, unifractal) [26], and 
wd950112 (monochrome version, computer-generated, multifractal) [31]. 

The cross from the intensity values, whose shanks meet in the examined point of the original 
image [1], was chosen as the first local surroundings. In contrast to the global recalculation, such a 
transformation of the texmos2.s512 image produces a substantially much richer intensity T^ ^-image. 
One can see that relatively simple global information consists of more complex local information 
(Figure 4a,c,e). 

However, the cross-local t5q5e of the image transformation is the least suitable approach for the 
analysis of the photograph of the jelly beans (Figure 5c). In this case, a circular local element is 
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recommended to be used instead. As seen in Figure 5d-f, the increase of the diameter up to the size 
of fhe jelly beans reduces fhe background gradually. The nexf increase enables grouping fhe jelly 
beans info higher-order assemblies. A similar grouping is observable for fhe smallesf squares in fhe 
fransformed fexmos2.s512 using fhe 29 px square surroundings (Figure 3f). In confrasf, lower values 
of square surroundings (Figure 3d) highlighf only fhe border infensifies. 


Algorithm 2 : Point information gain matrix (r^), point information gain entropy (Ha), and 
point information gain entropy density (E^) calculations for local kinds of information. 
Parameters a and b are semiaxes of fhe ellipse surroundings and a half-widfh of fhe recfangle 
surroundings, respecfively, a = 0 and b = 0 for fhe cross surroundings. 

Input: 2D discrete data Imxn', ot, where a > 0 A a 7 ^ 1; parameters of surroundings a, b 

Output: Ha, E^ 


1 Ia= zeros(I); % create a zero matrix Ta of the size of the Imxn matrix 

2 hashMap = containers.Map; % declare an empty hash-map (the key-value array) 


3 for f = (fl -|- 1 ) to (m — fl — 1 ) do 

4 for 7 = (b-F 1 ) to (n — b — 1 ) do 

5 h = getHist(I(i,y)); 

% create a histogram hfrom the elements around the pixel (i,j) of Imxn 

6 p = h/ sum(h); % explain the histogram has a probability histogram p 

7 h(I(f,;)) = h(I(i,7)) — 1 ; % remove the examined point (i,]) from the histogram h 

8 p2 = h/sum(h); % explain the histogram has a probability histogram p2 

9 = Y^log2(sum(p2.^a)/sum(p.^a)); 

% calculate F^'^^ as a difference of two Renyi entropies - with and without the examined 
element [i,]) (Equation (7)) 


10 

11 


V = I(f,y); % read a value of the element (intensity) at the position 1(1,;) 

checksum = calcCheckSum(h, z^); 

% calculate checksum using a hash-function effective enough (MD4, MD5, SHAl,...) 


13 


if not hashMap.isKeyfcteckSiim) then 
hashMap (checksum) = ra(f,y); 

% if the hash-map does not contain the key, insert a new element with the key checksum, 
where the inserted value is the Tccfor the element 1(1,]) 


end 


15 


16 end 


end 


17 Ha = sum(sum(ra)); % calculate Ha as a sum of all elements in the matrix F^ (Equation (19)) 

18 Ea = sum(values(hashMap)); % calculate E^ as a sum of all elements in the matrix 
hashMap (Equation (20)) 


3.3. Point Information Gain Entropy and Point Information Gain Entropy Density 

From the point of view of fhermod 5 mamics, fhe Ha and E^ can be considered as additive, 
homological sfafe variables whose knowledge can be helpful in analysis of mulfidimensional (image) 
dafa as well [32]. Despife fhe relafive familiarify of fheir formulas (Section 2.3), fhe Ha can be defined 
as a sum of all information contributions to the data distribution, either the global or partial one, 
i.e., all T^a \ whereas the Ea is a sum of all information microsfafes of fhe disfribufion. Even in case 
of fhe local information, each two (collision) hisfograms wifh fhe same proporfional represenfafion 
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of frequencies of elemenfs, which were obtained from disfribufions around fwo pixels af differenf 
posifions and only differing in fhe posifions of frequencies in fhe hisfogram, are considered fo be 
unique microsfates and produce unique values ri*^ (see Algorifhm 2). Thus, in agreement with the 
predictions arising from Equations (19) and (20), the -calculation does not suppress contributions 
of elemenfs with low probabilities of occurrences (rare poinfs) and is more robusf and sfable againsf 
changes in the local surroundings. This phenomenon manifests itself in fhe lower differences 
in dependencies Zx{oc) for four square surroundings in comparison fo fhe dependencies Hx{oc) in 
Figure 6. Neverfheless, if is worfh noting fhaf, during fhe calculafion wifh fhe usage of fhe local 
geomefrical surroundings, fhe surroundings fouch fhe edges of fhe image af mosf and only an inferior 
part of fhe image is processed. This facf—fechnical limifafion—^negafively influences values Ha and 
Ea for square surroundings in Figure 6 and also leads fo fhe lower sizes of T^ ^-fransformed images 
(e.g.. Figures 3d-f and 5d-f). 




a 



a 




oWhole 


a 


a 


■ Cross A Square 5 px a Square 11 px Square 15 px a Square 29 px 


Figure 6. Spectra Ha and Sa for global information and different local surroundings of a 
unifractal (texmos2.s512 [26], column (a)) and multifracal (wd950112 [31], column (b)) image at 
a = {0.1,0.2,..., 0.9,0.99,1.1,1.2, ...,4.0}. 
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Plotting the and vs. a in Figure 6 is not random. As mentioned for calculations 
(Section 2.1), multidimensional discrete (image) data is suitable to be characterized not only by one 
discrete value, either or E^, at a particular a, but also by their a-dependent spectra. The reason 
is not only to avoid digital rounding, but also to possibly to characterize the type and the origin 
of geomefrical sfrucfures in fhe image (cf. Secfion 3.1). Anofher application has been found in fhe 
sfafisfical evaluafion (clusfering) of fhe fime-lapse mulfidimensional dafasefs [32,33]. This calculation 
mefhod was originally developed for sfudy of mulfifracfal self-organizing biological images [34,35]; 
however, if enables description of any f 5 rpes of images. Since parfs of an image are forms of 
complex sfrucfures, fhe besf way fo inferpref fhe image is fo use a combination of ifs global and local 
kinds of informafion. We demonsfrafe fhis facf on an example of a unifracfal (almosf non-fracfal) 
Euclidian image and a compufer-generafed mulfifracfal image (Figure 6). Whereas fhe Euclidian 
image gives monofone specfra (for fhe global and cross-local kinds of informafion, even 

linear dependencies af fhe particular discrefe inferval of values a), fhe recalculation of fhe mulfifracfal 
image shows exfremes af values of a close fo 1. Analogous dependences were also plotted for fhe 
image sefs of fhe course of fhe self-organizing Belousov-Zhabofinsky reaction [32]. 

4. Materials and Methods 


4.1. Processing of Images and Typical Histograms 

The values of T^^ H^, and E^ for all fypical hisfograms and images were compufed using 
Equations (7), (19), and (20). Algorifhms are described in Secfion 4.2. The software and scripfs, as 
well as resulfs of all calculations, are available via ftp (Appendix). 

For the Cauchy, Levy and Gauss distributions, histograms of dependences of fhe number of 
elemenfs on fhe T^ were calculafed for a = {0.1, 0.3, 0.5, 0.7, 0.99,1.3, 1.5, 1.7, 2.0, 2.5, 3.0, 3.5, 4.0} 
using a Maflab® scrip!. The following probabilify densify funcfions f{x) were sfudied: 

(a) Levy disfribufion: 


f{x) = round 



\/lnx^ \ 


X e (1,256), X e N, c e {3,5,7}, 


(b) Cauchy disfribufion: 


f{x) = round 



1 

7r(l -F x^) 


X e (0,255), X e Z, c e {3.5,7}, 


(23) 


(24) 


(c) Gauss disfribufion: 


/(x)= round 



exp(-^) - 

ax/lfn 


X e (0,255), X e Z, c e {4,300} A a 


1, c e {3,4} A cr = 10. 


(25) 

In Figures 1 and 2, fhe Cauchy and Levy disfribufions wifh c = 7 and fhe Gauss disfribufion wifh 
paramefers c = 4 and a = 10 are depicfed. 

Mulfidimensional image analysis based on calculafion of T^'^ Ha, and E^ was fesfed on 5 
sfandard 8-bpc images (Table 1). Before fhe compufafions, original images wd950112.gif and 
6ASCP011.gif obfained from [31] were fransformed info monochrome “^.png formafs in Maflab® 
software. All images were processed using an Image Info Exfracfor Professional software (Insfifufe of 
Complex Sysfem, Universify of Soufh Bohemia, Nove Hrady Czech Republic) for a = {0.1,0.2,..., 0.9, 
0.99, 1.1, 1.2, ..., 4.0}. The global informafion was exfracfed using (fhe ifalics refer fo paramefers 
which are sef in fhe Image Info Exfracfor Professional software.) Whole Image calculafion. The 
verfical-horizonfal cross, square (a side of 5, 11, 15, and 29 px, respecfively), and circle (a radius 
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of 2,5, and 8 px, respectively) for local information were set as special cases of a Cross, Rectangle, and 
Ellipse calculation at the rotation angle Phi of 0°. Into the Image Info Extractor Professional software, 
a side of the square and radius of the circle surroundings was input as zvidth/2 and height/2 of 2, 5, 
and 14 px and a and b of 2,5, and 8 px, respectively. 

Table 1. Specifications of images. 


Image 

Source 

Colors 

Resolution 

Geometry 

Origin 

texmos2.s512.png 

[26] 

mono 

512x512 

unifractal 

computer-based 

4.1.07. tiff 

[26] 

RGB 

256x256 

unifractal 

photograph 

wash-ir.tiff 

[26] 

RGB 

2250x2250 

unifractal 

computer-based 

wd950112.png 

[31] 

mono 

1024x768 

multifractal 

computer-based 

6ASCP011.png 

[36] 

mono 

1600x1200 

multifractal 

computer-based 


4.2. Calculation Algorithms 

The algorithms implemented into the Image Info Extractor Professional are described in 
Algorithms 1-2. In the case of RGB images, the algorithms were applied to each color channel. The 
values Ta ^ were visualized by a full rescaling into 8-bit resolution. Let us note that, for a = 1, the 
equations in lines 9 of both algorithms switch to the calculation of the Shannon entropy. 

5. Conclusions 

In this article, we propose novel information quantities—a point information gain (ri’^), a point 
information gain entropy (H^), and a point information gain entropy density (E^). We found a 
monotone dependency of the number of the elements of a given property in the set on T^^ The 
variables Ha and can be used as quantities in multidimensional datasets for the definition of 
the information context. Examination of local information in the distribution shows a potential for 
in-depth insight into formation of observed structures and patterns. This option can be practically 
utilized in acquisition of differently resolved variables in the dataset. The method enables avoiding 
cases where the number of occurrences of a certain event is the same, but ,in distribution in time, 
space or along any other variable, differ. In principle, the variables H^ and Ea are unique for 
each distribution but suffer from problems with digital precision of the computation. Therefore, 
we propose their a-dependent spectra as proper characteristics of any discrete distribution, e.g., for 
clustering of multidimensional datasets. 
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Appendix 

All processed data are available at [37] (for more details, see Section 4): 

1. Folder "Figures" contains subfolders with results of Ha, and E^ calculations for "RGB" 
(4.1.07.tiff, wash-ir.tiff) and "gray" (texmos2.s512.png, wd950112.png, 6ASGP011.png) standard 
images calculated for 40 values a. The results are separated into subfolders according to the t 5 rpe 
of extracted information. 
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2. Folder "H_Xi" stores the PIE_PIED.xlsx and PIE_PIED2.xlsx files with dependencies of and 
Eft on a as exporfed from fhe PIE.maf files (in folder "Eigures"). Tifles of fhe graphs, which are 
in agreemenf wifh fhe compufed variables and exfracfed kinds of informafion, are wriffen in 
fhe sheefs. 

3. Eolder "Histograms" sfores fhe hisfograms of fhe occurrences of values for fhe Cauchy 
(two t 5 ^es). Levy (three types), and Gauss (four f 5 ^es) disfribufions. The paramefers of fhe 
original disfribufions are saved in fhe equafion.fxf files. All hisfograms were recalculafed using 
13 values a. 

4. Eolder "Software" confains a 32- and 64-bif version of an Image Info Exfracfor Professional v. b9 
soffware (ImageExfracfor_b9_xxbif.zip; supported by OS Win7) and a pig_hisfograms.m Maflab® 
scrip! for recalculafion of fhe f 5 rpical probabilify densify functions. A scrip! pie_ec.m serves for 
fhe exfracfion of Ha and E^ from fhe folders (oufpufs from fhe Image Info Exfracfor Professional) 
over a. In fhe soffware and scrip!, fhe variables P^^, H^, and E^ are called PIG, PIE, and PIED, 
respectively Manuals for fhe soffware and scripfs are also attached. 
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